{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "268d7741",
   "metadata": {},
   "source": [
    "(load-project-yaml-from-git)=\n",
    "# Load project YAML from Git, Zip, Tar source"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f263391c",
   "metadata": {},
   "source": [
    "After you create your project and you have a project.yaml file with all the necessery metadata within the remote source (Git, zip or gz.tar file), you can simply load that project and run, build, and deploy your functions and workflows.\n",
    "\n",
    "Run the project automation in {ref}`automate-project-git-source` before you run this workbook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48e13297",
   "metadata": {},
   "source": [
    "This notebook presents the steps to load a CI/CD project in MLRun:\n",
    "1. [Loading a project from a remote URL](#loading-a-project-from-a-remote-url)\n",
    "2. [Getting a function object](#getting-a-function-object)\n",
    "3. [Running project functions](#running-project-functions)\n",
    "5. [Deploying project functions](#deploying-project-functions)\n",
    "6. [Running the project workflow](#running-the-project-workflow)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5ca2246",
   "metadata": {},
   "source": [
    "Install mlrun using ``pip install mlrun==<mlrun server version>`` or ``sh align_mlrun.sh`` (the default mlrun installer that automatically installs the server version)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0bcb3a5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlrun"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea823444",
   "metadata": {},
   "source": [
    "## Loading a project from a remote URL\n",
    "\n",
    "This method can be used for loading an MLRun project from yaml/zip/tar/git/dir or from the MLRun DB."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ae2174b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# project source to load from -'git://url/org/repo.git#<branch-name or refs/heads/.. or refs/tags/..>`.\n",
    "source = \"git://github.com/mlrun/ci-cd-tutorial.git#refs/tags/v3\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d3bc0fe",
   "metadata": {},
   "source": [
    "**Note -** Add the git branch or refs to the source e.g.: 'git://<url>/org/repo.git#<branch-name or refs/heads/..>'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bf0977b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the project\n",
    "project = mlrun.load_project(\n",
    "    \"./clone\", url=source, clone=True, name=\"my-load-proj\", user_project=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94de1a60",
   "metadata": {},
   "source": [
    "For example:\n",
    "```\n",
    "# when loading from private repo\n",
    "project = mlrun.get_or_create_project(name='new-ci-cd-proj',context='./',init_git=True,secrets={\"GIT_TOKEN\":<github-token>})\n",
    "# when running functions in a project from a private repo\n",
    "project.set_secrets({\"GIT_TOKEN\":<github-token>}\n",
    "```\n",
    "\n",
    "See {py:class}`mlrun.projects.load_project`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d1dd9bc7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "kind: project\n",
      "metadata:\n",
      "  name: my-load-proj-shapira\n",
      "  created: '2023-04-17T13:27:10.756000'\n",
      "spec:\n",
      "  functions:\n",
      "  - url: ./src/data_fetch.py\n",
      "    name: data-fetch\n",
      "    kind: job\n",
      "    image: mlrun/mlrun\n",
      "    handler: data_fetch\n",
      "    with_repo: true\n",
      "    tag: v2\n",
      "  - url: ./src/train.py\n",
      "    name: train\n",
      "    kind: job\n",
      "    image: mlrun/mlrun\n",
      "    handler: train\n",
      "    with_repo: true\n",
      "    tag: v2\n",
      "  - url: ./function_spec/serving.yaml\n",
      "    name: serving\n",
      "  workflows:\n",
      "  - path: ./src/workflow.py\n",
      "    name: main\n",
      "  artifacts:\n",
      "  - kind: model\n",
      "    metadata:\n",
      "      project: new-ci-cd-proj-shapira\n",
      "      key: model-test\n",
      "    spec:\n",
      "      target_path: v3io:///projects/new-ci-cd-proj-shapira/artifacts/a5d545c6-fd5d-44e8-966c-24b9261314be/train/0/model/\n",
      "      model_file: model.pkl\n",
      "    status:\n",
      "      state: created\n",
      "  conda: ''\n",
      "  source: git://github.com/GiladShapira94/example-ci-cd.git#refs/heads/v2\n",
      "  origin_url: git://github.com/GiladShapira94/example-ci-cd.git#refs/heads/v2\n",
      "  load_source_on_run: true\n",
      "  desired_state: online\n",
      "status:\n",
      "  state: online\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# print project yaml\n",
    "print(project.to_yaml())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4676c527",
   "metadata": {},
   "source": [
    "## Getting a function object \n",
    "Get the function object using the {py:class}`~mlrun.projects.MlrunProject.get_function` method.\n",
    "\n",
    "This method allows you to get a function object based on the metadata in your project YAML file or from MLRun DB.\n",
    "````\n",
    "serving_func = project.get_function('<function name>')\n",
    "````"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "37743f5b",
   "metadata": {},
   "outputs": [],
   "source": [
    "serving_func = project.get_function(\"serving\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "035cf688",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<mlrun.serving.states.TaskStep at 0x7f7f88ba3410>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "serving_func.add_model(\n",
    "    key=\"model\",\n",
    "    model_path=train_run.outputs[\"model\"],\n",
    "    class_name=\"mlrun.frameworks.sklearn.SklearnModelServer\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8acbf7c2",
   "metadata": {},
   "source": [
    "```{admonition} Tip: Changing the model file path\n",
    "This serving function points to a model file whose path is stored in the function spec. If you want to change it (for example, to use a newer model file) you need to add the model to the function object and then deploy the function, or alternately, change the function.yaml in the remote source:\n",
    "\n",
    "\n",
    "      serving_func = project.get_function('serving')\n",
    "      serving_func.add_model(key='model',model_path=train_run.outputs[\"model\"],\n",
    "      class_name='mlrun.frameworks.sklearn.SklearnModelServer')\n",
    "      serving_dep = project.deploy_function('serving')\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89ea2afb",
   "metadata": {},
   "source": [
    "Test your serving function locally before deploying it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0ecd1cf6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:19:19,976 [warning] run command, file or code were not specified\n",
      "> 2023-05-17 09:19:20,579 [info] model model was loaded\n",
      "> 2023-05-17 09:19:20,580 [info] Loaded ['model']\n"
     ]
    }
   ],
   "source": [
    "serving_server = serving_func.to_mock_server()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "bd901ebd",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_data = \"\"\"{\"inputs\":[[-0.60150011,  0.51150308,  0.25701239, -1.51777297, -1.82961288,\n",
    "         0.22983693, -0.40761625,  0.82325082,  1.1779216 ,  1.08424275,\n",
    "        -0.7031145 , -0.40608979, -0.36305977,  1.28075006,  0.94445967,\n",
    "         1.19105828,  1.93498414,  0.69911167,  0.50759757,  0.91565635]]}\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "fc3766cc",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'id': '70c310d8fc10420fa9887546623b0ee0',\n",
       " 'model_name': 'model',\n",
       " 'outputs': [1]}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "serving_server.test(\"/\", my_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a98c2ddc",
   "metadata": {},
   "source": [
    "## Running project functions \n",
    "Run the function using the {py:class}`~mlrun.projects.MlrunProject.run_function` method both to \n",
    "[run jobs locally](./automate-project-git-source.html#running-the-function-locally) \n",
    "and, run remotely on the [runtime/cluster](./automate-project-git-source.html#running-the-function-remotely-on-your-cluster). If \n",
    "there are any requirements you need to build a new \n",
    "image before you run a function. See more details in {ref}`build-function-image`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e9c7fdd5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:15:38,824 [info] Storing function: {'name': 'data-fetch-data-fetch', 'uid': '5bd1b1e535894b1385ed1d6d33180741', 'db': 'http://mlrun-api:8080'}\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".dictlist {\n",
       "  background-color: #4EC64B;\n",
       "  text-align: center;\n",
       "  margin: 4px;\n",
       "  border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;}\n",
       ".artifact {\n",
       "  cursor: pointer;\n",
       "  background-color: #4EC64B;\n",
       "  text-align: left;\n",
       "  margin: 4px; border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;\n",
       "}\n",
       "div.block.hidden {\n",
       "  display: none;\n",
       "}\n",
       ".clickable {\n",
       "  cursor: pointer;\n",
       "}\n",
       ".ellipsis {\n",
       "  display: inline-block;\n",
       "  max-width: 60px;\n",
       "  white-space: nowrap;\n",
       "  overflow: hidden;\n",
       "  text-overflow: ellipsis;\n",
       "}\n",
       ".master-wrapper {\n",
       "  display: flex;\n",
       "  flex-flow: row nowrap;\n",
       "  justify-content: flex-start;\n",
       "  align-items: stretch;\n",
       "}\n",
       ".master-tbl {\n",
       "  flex: 3\n",
       "}\n",
       ".master-wrapper > div {\n",
       "  margin: 4px;\n",
       "  padding: 10px;\n",
       "}\n",
       "iframe.fileview {\n",
       "  border: 0 none;\n",
       "  height: 100%;\n",
       "  width: 100%;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       ".pane-header-title {\n",
       "  width: 80%;\n",
       "  font-weight: 500;\n",
       "}\n",
       ".pane-header {\n",
       "  line-height: 1;\n",
       "  background-color: #4EC64B;\n",
       "  padding: 3px;\n",
       "}\n",
       ".pane-header .close {\n",
       "  font-size: 20px;\n",
       "  font-weight: 700;\n",
       "  float: right;\n",
       "  margin-top: -5px;\n",
       "}\n",
       ".master-wrapper .right-pane {\n",
       "  border: 1px inset silver;\n",
       "  width: 40%;\n",
       "  min-height: 300px;\n",
       "  flex: 3\n",
       "  min-width: 500px;\n",
       "}\n",
       ".master-wrapper * {\n",
       "  box-sizing: border-box;\n",
       "}\n",
       "</style><script>\n",
       "function copyToClipboard(fld) {\n",
       "    if (document.queryCommandSupported && document.queryCommandSupported('copy')) {\n",
       "        var textarea = document.createElement('textarea');\n",
       "        textarea.textContent = fld.innerHTML;\n",
       "        textarea.style.position = 'fixed';\n",
       "        document.body.appendChild(textarea);\n",
       "        textarea.select();\n",
       "\n",
       "        try {\n",
       "            return document.execCommand('copy'); // Security exception may be thrown by some browsers.\n",
       "        } catch (ex) {\n",
       "\n",
       "        } finally {\n",
       "            document.body.removeChild(textarea);\n",
       "        }\n",
       "    }\n",
       "}\n",
       "function expandPanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName');\n",
       "  console.log(el.title);\n",
       "\n",
       "  document.querySelector(panelName + \"-title\").innerHTML = el.title\n",
       "  iframe = document.querySelector(panelName + \"-body\");\n",
       "\n",
       "  const tblcss = `<style> body { font-family: Arial, Helvetica, sans-serif;}\n",
       "    #csv { margin-bottom: 15px; }\n",
       "    #csv table { border-collapse: collapse;}\n",
       "    #csv table td { padding: 4px 8px; border: 1px solid silver;} </style>`;\n",
       "\n",
       "  function csvToHtmlTable(str) {\n",
       "    return '<div id=\"csv\"><table><tr><td>' +  str.replace(/[\\n\\r]+$/g, '').replace(/[\\n\\r]+/g, '</td></tr><tr><td>')\n",
       "      .replace(/,/g, '</td><td>') + '</td></tr></table></div>';\n",
       "  }\n",
       "\n",
       "  function reqListener () {\n",
       "    if (el.title.endsWith(\".csv\")) {\n",
       "      iframe.setAttribute(\"srcdoc\", tblcss + csvToHtmlTable(this.responseText));\n",
       "    } else {\n",
       "      iframe.setAttribute(\"srcdoc\", this.responseText);\n",
       "    }\n",
       "    console.log(this.responseText);\n",
       "  }\n",
       "\n",
       "  const oReq = new XMLHttpRequest();\n",
       "  oReq.addEventListener(\"load\", reqListener);\n",
       "  oReq.open(\"GET\", el.title);\n",
       "  oReq.send();\n",
       "\n",
       "\n",
       "  //iframe.src = el.title;\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.remove(\"hidden\");\n",
       "  }\n",
       "}\n",
       "function closePanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName')\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (!resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.add(\"hidden\");\n",
       "  }\n",
       "}\n",
       "\n",
       "</script>\n",
       "<div class=\"master-wrapper\">\n",
       "  <div class=\"block master-tbl\"><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>project</th>\n",
       "      <th>uid</th>\n",
       "      <th>iter</th>\n",
       "      <th>start</th>\n",
       "      <th>state</th>\n",
       "      <th>name</th>\n",
       "      <th>labels</th>\n",
       "      <th>inputs</th>\n",
       "      <th>parameters</th>\n",
       "      <th>results</th>\n",
       "      <th>artifacts</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>my-load-proj-shapira</td>\n",
       "      <td><div title=\"5bd1b1e535894b1385ed1d6d33180741\"><a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor/5bd1b1e535894b1385ed1d6d33180741/overview\" target=\"_blank\" >...33180741</a></div></td>\n",
       "      <td>0</td>\n",
       "      <td>May 17 09:15:38</td>\n",
       "      <td>completed</td>\n",
       "      <td>data-fetch-data-fetch</td>\n",
       "      <td><div class=\"dictlist\">v3io_user=shapira</div><div class=\"dictlist\">kind=</div><div class=\"dictlist\">owner=shapira</div><div class=\"dictlist\">host=jupyter-shapira-7fc985f9db-cp8x9</div><div class=\"dictlist\">release=v2</div></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td><div title=\"v3io:///projects/my-load-proj-shapira/artifacts/data-fetch-data-fetch/0/train-dataset.parquet\">train-dataset</div><div title=\"v3io:///projects/my-load-proj-shapira/artifacts/data-fetch-data-fetch/0/test-dataset.parquet\">test-dataset</div></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div></div>\n",
       "  <div id=\"result0f7b7c36-pane\" class=\"right-pane block hidden\">\n",
       "    <div class=\"pane-header\">\n",
       "      <span id=\"result0f7b7c36-title\" class=\"pane-header-title\">Title</span>\n",
       "      <span onclick=\"closePanel(this)\" paneName=\"result0f7b7c36\" class=\"close clickable\">&times;</span>\n",
       "    </div>\n",
       "    <iframe class=\"fileview\" id=\"result0f7b7c36-body\"></iframe>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<b> > to track results use the .show() or .logs() methods  or <a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor/5bd1b1e535894b1385ed1d6d33180741/overview\" target=\"_blank\">click here</a> to open in UI</b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:15:42,712 [info] run executed, status=completed: {'name': 'data-fetch-data-fetch'}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<mlrun.model.RunObject at 0x7f7f53862790>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "project.run_function(\n",
    "    function=\"data-fetch\", local=True, returns=[\"train-dataset\", \"test-dataset\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ba32d177",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:15:42,766 [info] Storing function: {'name': 'data-fetch-data-fetch', 'uid': 'bb814e47e2cd433b8820f19c782fb8af', 'db': 'http://mlrun-api:8080'}\n",
      "> 2023-05-17 09:15:43,048 [info] Job is running in the background, pod: data-fetch-data-fetch-q774n\n",
      "final state: completed\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".dictlist {\n",
       "  background-color: #4EC64B;\n",
       "  text-align: center;\n",
       "  margin: 4px;\n",
       "  border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;}\n",
       ".artifact {\n",
       "  cursor: pointer;\n",
       "  background-color: #4EC64B;\n",
       "  text-align: left;\n",
       "  margin: 4px; border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;\n",
       "}\n",
       "div.block.hidden {\n",
       "  display: none;\n",
       "}\n",
       ".clickable {\n",
       "  cursor: pointer;\n",
       "}\n",
       ".ellipsis {\n",
       "  display: inline-block;\n",
       "  max-width: 60px;\n",
       "  white-space: nowrap;\n",
       "  overflow: hidden;\n",
       "  text-overflow: ellipsis;\n",
       "}\n",
       ".master-wrapper {\n",
       "  display: flex;\n",
       "  flex-flow: row nowrap;\n",
       "  justify-content: flex-start;\n",
       "  align-items: stretch;\n",
       "}\n",
       ".master-tbl {\n",
       "  flex: 3\n",
       "}\n",
       ".master-wrapper > div {\n",
       "  margin: 4px;\n",
       "  padding: 10px;\n",
       "}\n",
       "iframe.fileview {\n",
       "  border: 0 none;\n",
       "  height: 100%;\n",
       "  width: 100%;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       ".pane-header-title {\n",
       "  width: 80%;\n",
       "  font-weight: 500;\n",
       "}\n",
       ".pane-header {\n",
       "  line-height: 1;\n",
       "  background-color: #4EC64B;\n",
       "  padding: 3px;\n",
       "}\n",
       ".pane-header .close {\n",
       "  font-size: 20px;\n",
       "  font-weight: 700;\n",
       "  float: right;\n",
       "  margin-top: -5px;\n",
       "}\n",
       ".master-wrapper .right-pane {\n",
       "  border: 1px inset silver;\n",
       "  width: 40%;\n",
       "  min-height: 300px;\n",
       "  flex: 3\n",
       "  min-width: 500px;\n",
       "}\n",
       ".master-wrapper * {\n",
       "  box-sizing: border-box;\n",
       "}\n",
       "</style><script>\n",
       "function copyToClipboard(fld) {\n",
       "    if (document.queryCommandSupported && document.queryCommandSupported('copy')) {\n",
       "        var textarea = document.createElement('textarea');\n",
       "        textarea.textContent = fld.innerHTML;\n",
       "        textarea.style.position = 'fixed';\n",
       "        document.body.appendChild(textarea);\n",
       "        textarea.select();\n",
       "\n",
       "        try {\n",
       "            return document.execCommand('copy'); // Security exception may be thrown by some browsers.\n",
       "        } catch (ex) {\n",
       "\n",
       "        } finally {\n",
       "            document.body.removeChild(textarea);\n",
       "        }\n",
       "    }\n",
       "}\n",
       "function expandPanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName');\n",
       "  console.log(el.title);\n",
       "\n",
       "  document.querySelector(panelName + \"-title\").innerHTML = el.title\n",
       "  iframe = document.querySelector(panelName + \"-body\");\n",
       "\n",
       "  const tblcss = `<style> body { font-family: Arial, Helvetica, sans-serif;}\n",
       "    #csv { margin-bottom: 15px; }\n",
       "    #csv table { border-collapse: collapse;}\n",
       "    #csv table td { padding: 4px 8px; border: 1px solid silver;} </style>`;\n",
       "\n",
       "  function csvToHtmlTable(str) {\n",
       "    return '<div id=\"csv\"><table><tr><td>' +  str.replace(/[\\n\\r]+$/g, '').replace(/[\\n\\r]+/g, '</td></tr><tr><td>')\n",
       "      .replace(/,/g, '</td><td>') + '</td></tr></table></div>';\n",
       "  }\n",
       "\n",
       "  function reqListener () {\n",
       "    if (el.title.endsWith(\".csv\")) {\n",
       "      iframe.setAttribute(\"srcdoc\", tblcss + csvToHtmlTable(this.responseText));\n",
       "    } else {\n",
       "      iframe.setAttribute(\"srcdoc\", this.responseText);\n",
       "    }\n",
       "    console.log(this.responseText);\n",
       "  }\n",
       "\n",
       "  const oReq = new XMLHttpRequest();\n",
       "  oReq.addEventListener(\"load\", reqListener);\n",
       "  oReq.open(\"GET\", el.title);\n",
       "  oReq.send();\n",
       "\n",
       "\n",
       "  //iframe.src = el.title;\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.remove(\"hidden\");\n",
       "  }\n",
       "}\n",
       "function closePanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName')\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (!resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.add(\"hidden\");\n",
       "  }\n",
       "}\n",
       "\n",
       "</script>\n",
       "<div class=\"master-wrapper\">\n",
       "  <div class=\"block master-tbl\"><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>project</th>\n",
       "      <th>uid</th>\n",
       "      <th>iter</th>\n",
       "      <th>start</th>\n",
       "      <th>state</th>\n",
       "      <th>name</th>\n",
       "      <th>labels</th>\n",
       "      <th>inputs</th>\n",
       "      <th>parameters</th>\n",
       "      <th>results</th>\n",
       "      <th>artifacts</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>my-load-proj-shapira</td>\n",
       "      <td><div title=\"bb814e47e2cd433b8820f19c782fb8af\"><a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor/bb814e47e2cd433b8820f19c782fb8af/overview\" target=\"_blank\" >...782fb8af</a></div></td>\n",
       "      <td>0</td>\n",
       "      <td>May 17 09:15:47</td>\n",
       "      <td>completed</td>\n",
       "      <td>data-fetch-data-fetch</td>\n",
       "      <td><div class=\"dictlist\">v3io_user=shapira</div><div class=\"dictlist\">kind=job</div><div class=\"dictlist\">owner=shapira</div><div class=\"dictlist\">mlrun/client_version=1.3.1-rc5</div><div class=\"dictlist\">mlrun/client_python_version=3.7.6</div><div class=\"dictlist\">host=data-fetch-data-fetch-q774n</div><div class=\"dictlist\">release=v2</div></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td><div title=\"v3io:///projects/my-load-proj-shapira/artifacts/data-fetch-data-fetch/0/train-dataset.parquet\">train-dataset</div><div title=\"v3io:///projects/my-load-proj-shapira/artifacts/data-fetch-data-fetch/0/test-dataset.parquet\">test-dataset</div></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div></div>\n",
       "  <div id=\"result037e7c0f-pane\" class=\"right-pane block hidden\">\n",
       "    <div class=\"pane-header\">\n",
       "      <span id=\"result037e7c0f-title\" class=\"pane-header-title\">Title</span>\n",
       "      <span onclick=\"closePanel(this)\" paneName=\"result037e7c0f\" class=\"close clickable\">&times;</span>\n",
       "    </div>\n",
       "    <iframe class=\"fileview\" id=\"result037e7c0f-body\"></iframe>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<b> > to track results use the .show() or .logs() methods  or <a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor/bb814e47e2cd433b8820f19c782fb8af/overview\" target=\"_blank\">click here</a> to open in UI</b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:15:56,204 [info] run executed, status=completed: {'name': 'data-fetch-data-fetch'}\n"
     ]
    }
   ],
   "source": [
    "data_fetch_run = project.run_function(\n",
    "    function=\"data-fetch\", local=False, returns=[\"train-dataset\", \"test-dataset\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6d206644",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:15:56,355 [info] Storing function: {'name': 'train-train', 'uid': 'b0b6137768c74af2b115b4399ee596e5', 'db': 'http://mlrun-api:8080'}\n",
      "> 2023-05-17 09:15:56,743 [info] Job is running in the background, pod: train-train-vzxw9\n",
      "final state: completed\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".dictlist {\n",
       "  background-color: #4EC64B;\n",
       "  text-align: center;\n",
       "  margin: 4px;\n",
       "  border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;}\n",
       ".artifact {\n",
       "  cursor: pointer;\n",
       "  background-color: #4EC64B;\n",
       "  text-align: left;\n",
       "  margin: 4px; border-radius: 3px; padding: 0px 3px 1px 3px; display: inline-block;\n",
       "}\n",
       "div.block.hidden {\n",
       "  display: none;\n",
       "}\n",
       ".clickable {\n",
       "  cursor: pointer;\n",
       "}\n",
       ".ellipsis {\n",
       "  display: inline-block;\n",
       "  max-width: 60px;\n",
       "  white-space: nowrap;\n",
       "  overflow: hidden;\n",
       "  text-overflow: ellipsis;\n",
       "}\n",
       ".master-wrapper {\n",
       "  display: flex;\n",
       "  flex-flow: row nowrap;\n",
       "  justify-content: flex-start;\n",
       "  align-items: stretch;\n",
       "}\n",
       ".master-tbl {\n",
       "  flex: 3\n",
       "}\n",
       ".master-wrapper > div {\n",
       "  margin: 4px;\n",
       "  padding: 10px;\n",
       "}\n",
       "iframe.fileview {\n",
       "  border: 0 none;\n",
       "  height: 100%;\n",
       "  width: 100%;\n",
       "  white-space: pre-wrap;\n",
       "}\n",
       ".pane-header-title {\n",
       "  width: 80%;\n",
       "  font-weight: 500;\n",
       "}\n",
       ".pane-header {\n",
       "  line-height: 1;\n",
       "  background-color: #4EC64B;\n",
       "  padding: 3px;\n",
       "}\n",
       ".pane-header .close {\n",
       "  font-size: 20px;\n",
       "  font-weight: 700;\n",
       "  float: right;\n",
       "  margin-top: -5px;\n",
       "}\n",
       ".master-wrapper .right-pane {\n",
       "  border: 1px inset silver;\n",
       "  width: 40%;\n",
       "  min-height: 300px;\n",
       "  flex: 3\n",
       "  min-width: 500px;\n",
       "}\n",
       ".master-wrapper * {\n",
       "  box-sizing: border-box;\n",
       "}\n",
       "</style><script>\n",
       "function copyToClipboard(fld) {\n",
       "    if (document.queryCommandSupported && document.queryCommandSupported('copy')) {\n",
       "        var textarea = document.createElement('textarea');\n",
       "        textarea.textContent = fld.innerHTML;\n",
       "        textarea.style.position = 'fixed';\n",
       "        document.body.appendChild(textarea);\n",
       "        textarea.select();\n",
       "\n",
       "        try {\n",
       "            return document.execCommand('copy'); // Security exception may be thrown by some browsers.\n",
       "        } catch (ex) {\n",
       "\n",
       "        } finally {\n",
       "            document.body.removeChild(textarea);\n",
       "        }\n",
       "    }\n",
       "}\n",
       "function expandPanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName');\n",
       "  console.log(el.title);\n",
       "\n",
       "  document.querySelector(panelName + \"-title\").innerHTML = el.title\n",
       "  iframe = document.querySelector(panelName + \"-body\");\n",
       "\n",
       "  const tblcss = `<style> body { font-family: Arial, Helvetica, sans-serif;}\n",
       "    #csv { margin-bottom: 15px; }\n",
       "    #csv table { border-collapse: collapse;}\n",
       "    #csv table td { padding: 4px 8px; border: 1px solid silver;} </style>`;\n",
       "\n",
       "  function csvToHtmlTable(str) {\n",
       "    return '<div id=\"csv\"><table><tr><td>' +  str.replace(/[\\n\\r]+$/g, '').replace(/[\\n\\r]+/g, '</td></tr><tr><td>')\n",
       "      .replace(/,/g, '</td><td>') + '</td></tr></table></div>';\n",
       "  }\n",
       "\n",
       "  function reqListener () {\n",
       "    if (el.title.endsWith(\".csv\")) {\n",
       "      iframe.setAttribute(\"srcdoc\", tblcss + csvToHtmlTable(this.responseText));\n",
       "    } else {\n",
       "      iframe.setAttribute(\"srcdoc\", this.responseText);\n",
       "    }\n",
       "    console.log(this.responseText);\n",
       "  }\n",
       "\n",
       "  const oReq = new XMLHttpRequest();\n",
       "  oReq.addEventListener(\"load\", reqListener);\n",
       "  oReq.open(\"GET\", el.title);\n",
       "  oReq.send();\n",
       "\n",
       "\n",
       "  //iframe.src = el.title;\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.remove(\"hidden\");\n",
       "  }\n",
       "}\n",
       "function closePanel(el) {\n",
       "  const panelName = \"#\" + el.getAttribute('paneName')\n",
       "  const resultPane = document.querySelector(panelName + \"-pane\");\n",
       "  if (!resultPane.classList.contains(\"hidden\")) {\n",
       "    resultPane.classList.add(\"hidden\");\n",
       "  }\n",
       "}\n",
       "\n",
       "</script>\n",
       "<div class=\"master-wrapper\">\n",
       "  <div class=\"block master-tbl\"><div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>project</th>\n",
       "      <th>uid</th>\n",
       "      <th>iter</th>\n",
       "      <th>start</th>\n",
       "      <th>state</th>\n",
       "      <th>name</th>\n",
       "      <th>labels</th>\n",
       "      <th>inputs</th>\n",
       "      <th>parameters</th>\n",
       "      <th>results</th>\n",
       "      <th>artifacts</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>my-load-proj-shapira</td>\n",
       "      <td><div title=\"b0b6137768c74af2b115b4399ee596e5\"><a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor/b0b6137768c74af2b115b4399ee596e5/overview\" target=\"_blank\" >...9ee596e5</a></div></td>\n",
       "      <td>0</td>\n",
       "      <td>May 17 09:16:02</td>\n",
       "      <td>completed</td>\n",
       "      <td>train-train</td>\n",
       "      <td><div class=\"dictlist\">v3io_user=shapira</div><div class=\"dictlist\">kind=job</div><div class=\"dictlist\">owner=shapira</div><div class=\"dictlist\">mlrun/client_version=1.3.1-rc5</div><div class=\"dictlist\">mlrun/client_python_version=3.7.6</div><div class=\"dictlist\">host=train-train-vzxw9</div><div class=\"dictlist\">release=v2</div></td>\n",
       "      <td><div title=\"store://artifacts/my-load-proj-shapira/data-fetch-data-fetch_train-dataset:bb814e47e2cd433b8820f19c782fb8af\">train_data</div><div title=\"store://artifacts/my-load-proj-shapira/data-fetch-data-fetch_test-dataset:bb814e47e2cd433b8820f19c782fb8af\">test_data</div></td>\n",
       "      <td></td>\n",
       "      <td><div class=\"dictlist\">accuracy=0.85</div><div class=\"dictlist\">f1_score=0.88</div><div class=\"dictlist\">precision_score=0.7857142857142857</div><div class=\"dictlist\">recall_score=1.0</div></td>\n",
       "      <td><div class=\"artifact\" onclick=\"expandPanel(this)\" paneName=\"result77eb53a7\" title=\"files/v3io/projects/my-load-proj-shapira/artifacts/train-train/0/feature-importance.html\">feature-importance</div><div title=\"v3io:///projects/my-load-proj-shapira/artifacts/train-train/0/test_set.parquet\">test_set</div><div class=\"artifact\" onclick=\"expandPanel(this)\" paneName=\"result77eb53a7\" title=\"files/v3io/projects/my-load-proj-shapira/artifacts/train-train/0/confusion-matrix.html\">confusion-matrix</div><div class=\"artifact\" onclick=\"expandPanel(this)\" paneName=\"result77eb53a7\" title=\"files/v3io/projects/my-load-proj-shapira/artifacts/train-train/0/roc-curves.html\">roc-curves</div><div class=\"artifact\" onclick=\"expandPanel(this)\" paneName=\"result77eb53a7\" title=\"files/v3io/projects/my-load-proj-shapira/artifacts/train-train/0/calibration-curve.html\">calibration-curve</div><div title=\"v3io:///projects/my-load-proj-shapira/artifacts/train-train/0/model/\">model</div></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div></div>\n",
       "  <div id=\"result77eb53a7-pane\" class=\"right-pane block hidden\">\n",
       "    <div class=\"pane-header\">\n",
       "      <span id=\"result77eb53a7-title\" class=\"pane-header-title\">Title</span>\n",
       "      <span onclick=\"closePanel(this)\" paneName=\"result77eb53a7\" class=\"close clickable\">&times;</span>\n",
       "    </div>\n",
       "    <iframe class=\"fileview\" id=\"result77eb53a7-body\"></iframe>\n",
       "  </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<b> > to track results use the .show() or .logs() methods  or <a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor/b0b6137768c74af2b115b4399ee596e5/overview\" target=\"_blank\">click here</a> to open in UI</b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:16:18,044 [info] run executed, status=completed: {'name': 'train-train'}\n"
     ]
    }
   ],
   "source": [
    "train_run = project.run_function(\n",
    "    function=\"train\",\n",
    "    inputs={\n",
    "        \"train_data\": data_fetch_run.outputs[\"train-dataset\"],\n",
    "        \"test_data\": data_fetch_run.outputs[\"test-dataset\"],\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "122cd04e",
   "metadata": {},
   "source": [
    "## Deploying project functions\n",
    "To deploy a remote function e.g. nuclio or serving function, use the {py:class}`~mlrun.projects.MlrunProject.deploy_function` method. \n",
    "You must use this method before invoking Nuclio or serving functions.\n",
    "````\n",
    "nuclio_func=project.deploy_function(function='<function name>')\n",
    "\n",
    "nuclio_func.function.invoke('/',{'int':4})\n",
    "````"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "89950a6e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:19:25,799 [info] Starting remote function deploy\n",
      "2023-05-17 09:19:26  (info) Deploying function\n",
      "2023-05-17 09:19:26  (info) Building\n",
      "2023-05-17 09:19:26  (info) Staging files and preparing base images\n",
      "2023-05-17 09:19:26  (info) Building processor image\n",
      "2023-05-17 09:20:41  (info) Build complete\n",
      "2023-05-17 09:21:19  (info) Function deploy complete\n",
      "> 2023-05-17 09:21:27,112 [info] successfully deployed function: {'internal_invocation_urls': ['nuclio-my-load-proj-shapira-serving-v2.default-tenant.svc.cluster.local:8080'], 'external_invocation_urls': ['my-load-proj-shapira-serving-v2-my-load-proj-shapira.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/']}\n"
     ]
    }
   ],
   "source": [
    "serving_dep = project.deploy_function(\"serving\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "8ea18d9d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:21:27,192 [info] invoking function: {'method': 'POST', 'path': 'http://nuclio-my-load-proj-shapira-serving-v2.default-tenant.svc.cluster.local:8080/'}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'id': 'efb4e274-00c2-428d-b462-92222bc64ce5',\n",
       " 'model_name': 'model',\n",
       " 'outputs': [1]}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "serving_dep.function.invoke(\"/\", my_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d1786c6",
   "metadata": {},
   "source": [
    "## Running the project workflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "1a89c5e6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>Pipeline running (id=b6ebe4fd-457e-4992-8eb5-a1b70fc44b94), <a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor-workflows/workflow/b6ebe4fd-457e-4992-8eb5-a1b70fc44b94\" target=\"_blank\"><b>click here</b></a> to view the details in MLRun UI</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.40.1 (20161225.0304)\n",
       " -->\n",
       "<!-- Title: kfp Pages: 1 -->\n",
       "<svg width=\"130pt\" height=\"188pt\"\n",
       " viewBox=\"0.00 0.00 130.00 188.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 184)\">\n",
       "<title>kfp</title>\n",
       "<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-184 126,-184 126,4 -4,4\"/>\n",
       "<!-- ci&#45;cd&#45;tutorial&#45;nplzh&#45;1091444859 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>ci&#45;cd&#45;tutorial&#45;nplzh&#45;1091444859</title>\n",
       "<polygon fill=\"#00ff00\" stroke=\"#000000\" points=\"122,-36 4,-36 0,-32 0,0 118,0 122,-4 122,-36\"/>\n",
       "<polyline fill=\"none\" stroke=\"#000000\" points=\"118,-32 0,-32 \"/>\n",
       "<polyline fill=\"none\" stroke=\"#000000\" points=\"118,-32 118,0 \"/>\n",
       "<polyline fill=\"none\" stroke=\"#000000\" points=\"118,-32 122,-36 \"/>\n",
       "<text text-anchor=\"middle\" x=\"61\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">deploy&#45;serving</text>\n",
       "</g>\n",
       "<!-- ci&#45;cd&#45;tutorial&#45;nplzh&#45;1597241585 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>ci&#45;cd&#45;tutorial&#45;nplzh&#45;1597241585</title>\n",
       "<ellipse fill=\"#00ff00\" stroke=\"#000000\" cx=\"61\" cy=\"-90\" rx=\"33.2948\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"61\" y=\"-86.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">train</text>\n",
       "</g>\n",
       "<!-- ci&#45;cd&#45;tutorial&#45;nplzh&#45;1597241585&#45;&gt;ci&#45;cd&#45;tutorial&#45;nplzh&#45;1091444859 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>ci&#45;cd&#45;tutorial&#45;nplzh&#45;1597241585&#45;&gt;ci&#45;cd&#45;tutorial&#45;nplzh&#45;1091444859</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M61,-71.8314C61,-64.131 61,-54.9743 61,-46.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"64.5001,-46.4132 61,-36.4133 57.5001,-46.4133 64.5001,-46.4132\"/>\n",
       "</g>\n",
       "<!-- ci&#45;cd&#45;tutorial&#45;nplzh&#45;604068056 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>ci&#45;cd&#45;tutorial&#45;nplzh&#45;604068056</title>\n",
       "<ellipse fill=\"#00ff00\" stroke=\"#000000\" cx=\"61\" cy=\"-162\" rx=\"57.6901\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"61\" y=\"-158.3\" font-family=\"Times,serif\" font-size=\"14.00\" fill=\"#000000\">data&#45;fetch</text>\n",
       "</g>\n",
       "<!-- ci&#45;cd&#45;tutorial&#45;nplzh&#45;604068056&#45;&gt;ci&#45;cd&#45;tutorial&#45;nplzh&#45;1597241585 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>ci&#45;cd&#45;tutorial&#45;nplzh&#45;604068056&#45;&gt;ci&#45;cd&#45;tutorial&#45;nplzh&#45;1597241585</title>\n",
       "<path fill=\"none\" stroke=\"#000000\" d=\"M61,-143.8314C61,-136.131 61,-126.9743 61,-118.4166\"/>\n",
       "<polygon fill=\"#000000\" stroke=\"#000000\" points=\"64.5001,-118.4132 61,-108.4133 57.5001,-118.4133 64.5001,-118.4132\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x7f7f7660c310>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h2>Run Results</h2><h3>[info] Workflow b6ebe4fd-457e-4992-8eb5-a1b70fc44b94 finished, state=Succeeded</h3><br>click the hyper links below to see detailed results<br><table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>uid</th>\n",
       "      <th>start</th>\n",
       "      <th>state</th>\n",
       "      <th>name</th>\n",
       "      <th>parameters</th>\n",
       "      <th>results</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td><div title=\"a2f802f351b3405db42e88abfe1ce62b\"><a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor/a2f802f351b3405db42e88abfe1ce62b/overview\" target=\"_blank\" >...fe1ce62b</a></div></td>\n",
       "      <td>May 17 09:22:14</td>\n",
       "      <td>completed</td>\n",
       "      <td>train</td>\n",
       "      <td></td>\n",
       "      <td><div class=\"dictlist\">accuracy=0.8</div><div class=\"dictlist\">f1_score=0.7999999999999999</div><div class=\"dictlist\">precision_score=0.7272727272727273</div><div class=\"dictlist\">recall_score=0.8888888888888888</div></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td><div title=\"8a03f422fdeb4935a4a6a0bddd17518b\"><a href=\"https://dashboard.default-tenant.app.cust-cs-il-3-5-2.iguazio-cd2.com/mlprojects/my-load-proj-shapira/jobs/monitor/8a03f422fdeb4935a4a6a0bddd17518b/overview\" target=\"_blank\" >...dd17518b</a></div></td>\n",
       "      <td>May 17 09:21:43</td>\n",
       "      <td>completed</td>\n",
       "      <td>data-fetch</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "b6ebe4fd-457e-4992-8eb5-a1b70fc44b94"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# run the workflow named main and wait for the pipeline completion (watch=True)\n",
    "project.run(\"main\", watch=True, engine=\"remote:kfp\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c300faa6",
   "metadata": {},
   "source": [
    "## Running a scheduled workflow\n",
    "\n",
    "For more information about scheduling workflows, see {ref}`scheduled-jobs`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "0130d760",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> 2023-05-17 09:24:14,370 [warning] WARNING!, you seem to have uncommitted git changes, use .push()\n",
      "> 2023-05-17 09:24:14,373 [info] executing workflow scheduling 'workflow-runner-main' remotely with kfp engine\n",
      "> 2023-05-17 09:24:14,377 [info] Storing function: {'name': 'main', 'uid': 'ff401cc316574c4ea94043ddcbab3a9e', 'db': 'http://mlrun-api:8080'}\n",
      "> 2023-05-17 09:24:14,966 [info] task schedule created: {'schedule': '0 * * * *', 'project': 'my-load-proj-shapira', 'name': 'main'}\n"
     ]
    }
   ],
   "source": [
    "project.run(\"main\", watch=True, schedule=\"0 * * * *\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
